Be Your Own Prada: Fashion Synthesis with Structural Coherence

摘要

We present a novel and effective approach for generating new clothing on awearer through generative adversarial learning. Given an input image of aperson and a sentence describing a different outfit, our model "redresses" theperson as desired, while at the same time keeping the wearer and her/his poseunchanged. Generating new outfits with precise regions conforming to a languagedescription while retaining wearer's body structure is a new challenging task.Existing generative adversarial networks are not ideal in ensuring globalcoherence of structure given both the input photograph and language descriptionas conditions. We address this challenge by decomposing the complex generativeprocess into two conditional stages. In the first stage, we generate aplausible semantic segmentation map that obeys the wearer's pose as a latentspatial arrangement. An effective spatial constraint is formulated to guide thegeneration of this semantic segmentation map. In the second stage, a generativemodel with a newly proposed compositional mapping layer is used to render thefinal image with precise regions and textures conditioned on this map. Weextended the DeepFashion dataset [8] by collecting sentence descriptions for79K images. We demonstrate the effectiveness of our approach through bothquantitative and qualitative evaluations. A user study is also conducted. Thecodes and the data are available at http://mmlab.ie.cuhk.edu.hk/projects/FashionGAN/.